Haptic feedback during phone calls

ABSTRACT

A method for providing haptic feedback to participants of multi-party phone conversations that includes opening a communications session with a conference system for at least two users each having user specific communications devices, user specific conduct measuring devices, and user specific haptic feedback device registered with the conference system. Analyzing content of the communications session from content received by the conference system through the user specific communications device for at least one of the users; and capturing status for said at least two users from data measured by the user specific conduct measuring device for the at least two users. Determining with the conference system if the content of the communications session and the status of said at least two users calls for input by the user through said user specific communications device. Sending a feedback signal from the conference system to the user specific haptic feedback device.

BACKGROUND Technical Field

The present invention generally relates to communications, and moreparticularly to managing feedback on multi-party phone calls.

Description of the Related Art

Historically, many meetings were face to face. Currently, there arenumerous techniques for conducting meetings, in a business setting orotherwise. Many business meetings have participants that are on “phone”calls or ‘conference calls’, using traditional phones, cellular phonesand/or voice over internet protocol (VOIP) using a computer device.During the calls, the people frequently put themselves on mute ormulti-task, which can lead to others not hearing what they are saying,or conversation existing that require them to refocus, yet they are notbeing attentive.

SUMMARY

In accordance with an embodiment of the present invention, a method ofproviding feedback to participants of multi-party phone conversations isprovided. In one embodiment, the method may be a computer implementedmethod for providing haptic feedback to participants of multi-partyphone conversations that includes opening a communications session witha conference system for at least two users, wherein each of the at leasttwo users has a user specific communications device registered with theconference system, a user specific conduct measuring device registeredwith the conference system, and a user specific haptic feedback deviceregistered with the conference system. Analyzing content of thecommunications session from content received by the conference systemthrough the user specific communications device for at least one of theusers; and capturing status for said at least two users from datameasured by the user specific conduct measuring device for the at leasttwo users. The method may further include determining with theconference system if the content of the communications session and thestatus of said at least two users calls for input by the user throughsaid user specific communications device. A feedback signal is sent fromthe conference system to the user specific haptic feedback device of theuser that the conference system has determined that the communicationssession has called for said input. The user specific haptic feedbackdevice deliverers the feedback signal to the user.

In another aspect, a system is provided for providing feedback toparticipants of multi-party phone conversations. In one embodiment, thesystem may include a registration database that includes a plurality ofuser accounts including addresses for user specific communicationsdevice registered with the conference system, a user specific conductmeasuring devices, and a user specific haptic feedback device. A contentinput recorder for receiving communications content from acommunications session from content received through the user specificcommunications device for at least one of the users. A conduct inputrecorder for recording a capturing status for said at least two usersfrom data measured by the user specific conduct measuring device. Anindividual status calculator including at least one hardware deviceprocessor for performing a set of instruction stored on at least onememory device, the individual status calculator analyzing the content ofthe communications session received from the user specificcommunications device from at least one users, analyzing the status ofsaid at least two users measured by the user specific conduct measuringdevice signals, and calculating that an individual status of one of saidat least two users calls for input through said user specificcommunications device by said user having the individual status callingfor said input. The system may further include a feedback signal to userspecific haptic feedback device transmitter that sends a signal forinput through said user specific communications device to a user havingthe individual status calling for said input. The user specific hapticfeedback device delivering the feedback signal to the user having theindividual status calling for said input.

In another aspect, the present disclosure provides a computer programproduct comprising a non-transitory computer readable storage mediumhaving computer readable program code embodied therein for providingfeedback to participants of multi-party phone conversations. The methodactuated by the computer program product may include providing hapticfeedback to participants of multi-party phone conversations thatincludes opening a communications session with a conference system forat least two users, wherein each of the at least two users has a userspecific communications device registered with the conference system, auser specific conduct measuring device registered with the conferencesystem, and a user specific haptic feedback device registered with theconference system. Analyzing content of the communications session fromcontent received by the conference system through the user specificcommunications device for at least one of the users; and capturingstatus for said at least two users from data measured by the userspecific conduct measuring device for the at least two users. The methodmay further include determining with the conference system if thecontent of the communications session and the status of said at leasttwo users calls for input by the user through said user specificcommunications device. A feedback signal is sent from the conferencesystem to the user specific haptic feedback device of the user that theconference system has determined that the communications session hascalled for said input. The user specific haptic feedback devicedeliverers the feedback signal to the user.

These and other features and advantages will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description will provide details of preferred embodimentswith reference to the following figures wherein:

FIG. 1 is a diagram illustrating an example environment for providingfeedback to participants of multi-party phone conversations of theirindividual status in the conversation, in accordance with one embodimentof the present disclosure.

FIG. 2 is a block/flow diagram showing a method of providing feedback toparticipants of multi-party phone conversations of their individualstatus in the conversation, in accordance with an embodiment of thepresent invention.

FIG. 3 is a block diagram illustrating a conferencing system forproviding feedback to participants of multi-party phone conversations oftheir individual status in the conversation, in accordance with oneembodiment of the present disclosure.

FIG. 4 is a block diagram illustrating a system that can incorporate theconferencing system depicted in FIG. 3 for providing feedback toparticipants of multi-party phone conversations of their individualstatus in the conversation, in accordance with one embodiment of thepresent disclosure.

FIG. 5 is a generalized diagram of a neural network.

FIG. 6 illustrates an example of an artificial neural network (ANN)architecture.

FIG. 7 is a block diagram of a neuron for use with the neural networkdepicted in FIG. 5 and the artificial neural network (ANN) architecturedepicted in FIG. 6.

FIG. 8 depicts a cloud computing environment according to an embodimentof the present disclosure.

FIG. 9 depicts abstraction model layers according to an embodiment ofthe present disclosure.

DETAILED DESCRIPTION

In some embodiments, the disclosure provides methods, systems andcomputer program products that provide a configurable haptic feedback toindividuals engaged in a call, e.g., telephone call. The term “haptic”denotes communications that create the sense of touch by applyingforces, vibrations, or motions to the user. In one example, the feedbackis activated when a person is talking for the meeting purpose, but hasaccidentally set his voice call device on mute, and no one can here theintended message. In this instance, a haptic device that is incommunication with the person who has accidentally set his voice calldevice on mute, is used to signal to that person speaking that his voicecall device is on mute, and that no one can here the intended message.In another example, the feedback is activated when a person is nottalking for the meeting purpose, yet all the other parties to theconversation are hearing this non-relevant information, because he hasnot muted his voice call device. In this instance, a haptic device thatis in communication with the person who does not realize his voice calldevice on mute, is used to signal to that person that non-relevantinformation is being transmitted to the multi-party communication overhis voice call device. Once receiving the haptic signal, the person canmute his or her voice call device. In yet another example, the hapticdevice is used to alert a person on a multi-party communication that thesubject of the communication is intended for them. The systems, methodsand computer program products are now described with more detail withreference to FIGS. 1-9.

FIG. 1 is a diagram illustrating an example environment for providingfeedback to participants of multi-party phone conversations of theirindividual status in the conversation. The example environment includesa network 104 over which users, i.e., 102 a, 102 b, 102 c, 102 d, cancommunication via a multi-party communication session, through which aconference system 106 can provide feedback to participants ofmulti-party phone conversations of their individual status in theconversation. In example embodiments, a user, e.g., user 102 a, 102 b,102 c, 102 d, participates on a call with one or more other users via acommunication device 102 that is communicatively coupled via the network104 to the conference system 106. Each user 102 a, 102 b, 102 c, 102 dmay include a communication device 53 a, 53 b, 53 c, 53 d through whichthey can listen to the conversation and participate with theconversation. Each user 102 a, 102 b, 102 c, 102 d may include an TOTdevice, in which the TOT device 52 a, 52 b, 52 c, 52 d has the abilityto measure the users conduct in determining whether their attention isdirected to the multi-party conversation, or whether their attention isnot directed to the multi-party conversation. The IOT devices 52 a, 52b, 52 c, 52 d are in communication with the conference system 106through the network. Each user 102 a, 102 b, 102 c, 102 d may alsoinclude a haptic feedback device 51 a, 51 b, 51 c, 51 d. The hapticfeedback device 51 a, 51 b, 51 c, 51 d is also in communication with theconference system 106 via the network 104. The haptic feedback device 51a, 51 b, 51 c, 51 d is the device that alerts the users 102 a, 102 b,102 c, 102 d, if their individual status in the conversation has changedthat would warrant a change in their attention to the multi-party phoneconversation.

The communication device 53 a, 53 b, 53 c, 54 d may comprise, but is notlimited to, a phone, a wearable phone, a smartphone, tablet, laptop,multi-processor system, microprocessor-based or programmable consumerelectronics, game console, set-top box, or any other device that theuser may utilize to communicate over the network 104. In someembodiments, the communication device 53 a, 53 b, 53 c, 54 d maycomprise a display module (not shown) to display information (e.g., inthe form of user interfaces). In further embodiments, the communicationdevice 53 a, 53 b, 53 c, 54 d may comprise one or more of a touchscreen, camera, keyboard, and microphone. During a communicationsession, the communication device 53 a, 53 b, 53 c, 54 d may be mutedand a mute detection analysis performed.

The Internet of things (IoT) is the network of physical devices,vehicles, home appliances, and other items embedded with electronics,software, sensors, actuators, and connectivity which enables thesethings to connect, collect and exchange data. The IoT devices 52 a, 52b, 52 c, 52 d can be equipped with various types of sensors to collectinformation about themselves and their surroundings, and provide thecollected information to the conference system 106 over the network 104.

In some examples, the IOT devices 52 a, 52 b, 52 c, 52 d or a hub thatthe IOT devices 52 a, 52 b, 52 c, 52 d are in communication with includea cellular radio to establish a connection to the Internet via acellular service such as a 4G (e.g., Mobile WiMAX, LTE) or 5G cellulardata service. Alternatively, or in addition, the IOT devices or a hubthat the IOT devices are in communication with include a may include aWiFi radio to establish a WiFi connection through a WiFi access point orrouter which couples the IOT devices 52 a, 52 b, 52 c, 52 d or IOT hubto the Internet (e.g., via an Internet Service Provider providingInternet service to the end user), which in turn connect to theconference system 106/network 104. Of course, it should be noted thatthe underlying principles of the invention are not limited to anyparticular type of communication channel or protocol.

In one embodiment, the IoT devices 52 a, 52 b, 52 c, 52 d are ultralow-power devices capable of operating for extended periods of time onbattery power (e.g., years). To conserve power, the local communicationchannels may be implemented using a low-power wireless communicationtechnology such as Bluetooth Low Energy (LE). In this embodiment, eachof the IoT devices 52 a, 52 b, 52 c, 52 d are equipped with Bluetooth LEradios and protocol stacks.

In one embodiment, the IoT platform includes an IoT app or Webapplication executed on user devices 102 a, 102 b, 102 c, 102 d to allowusers to access and configure the connected IoT devices 52 a, 52 b, 52c, 52 d, an IoT hub, and/or IoT service. The Web application may providefor communication over the network 104 to the conference system 106.

One form of internet of things (IOT) that is suitable for use as theconnected IoT devices 52 a, 52 b, 52 c, 52 d to the network 104 includesa microphone for measuring sounds, such as voices. These may be virtualassistant type devices. Examples of these types of devices may includeCortana™ by Microsoft, Apple Siri™, Google Assistant, Amazon Alexa™, andSamsung Bixby™ The Alexa virtual assistant type devices by Amazon aresupported by a number of hardware factors, such as speakers (smartspeakers), e.g., Amazon Echo™, Amazon Echo Plus™, Amazon Echo Dot™;televisions and media boxes; phones and tablet computers; laptop anddesktop computers; smart home devices, such as lamps, light switches,thermostats, smoke alarms etc.; wearable and earphones and computerprogram products. The Cortana™ virtual assistant provided by Microsoftcan be provided by computer operating systems such as Windows 10™,Windows 10 Mobile™, Windows Phone™, iOS™, Android™, etc. The Cortana™virtual assistance provided by Microsoft may also be available in anumber of hardware form factors, such as smart speakers, headphones,video game systems, e.g., Xbox One™, etc.

In some examples, the combination of the IoT devices 52 a, 52 b, 52 c,52 d, having a microphone for receiving user communications, i.e., forworking with the conference system 106, can include a system that has auniversal vocabulary, or basic machine-to-machine language capabilities,that can allow IoT devices 52 a, 52 b, 52 c, 52 d to communicate withthe conference system 106, for providing communications by the users tobe analyzed for their relevance to the subject of the multi-partyconversation. For example, the vocabulary measured by the IoT devices 52a, 52 b, 52 c, 52 d can be in plain English phrases or through someother form of abstraction. The vocabulary can be scalable such that newadditions can be added to the vocabulary, e.g. for more complex actions.In one example, the vocabulary may be, or may include, an STN model forIoT devices.

The IOT devices 52 a, 52 b, 52 c, 52 d may also include a video camerafor recording the motions of the users 102 a, 102 b, 102 c, 102 d. Thisvideo information can be used to determine if the user 102 a, 102 b, 102c, 102 d is engaged in an activity/exercise/event that would be takinghis attention away from the multi-party conversation. The IOT devices 52a, 52 b, 52 c, 52 d may be the same devices, or different devices fromthe IOT devices 52 a, 52 b, 52 c, 52 d including the microphone forcapturing sound and/or speech from the users 102 a, 102 b, 102 c, 102 d.For example, the IOT devices 52 a, 52 b, 52 c, 52 d that include a videocamera may be a computer-based hardware device that includes aprocessor, memory, and communication capabilities. Each of the IOTdevices including the camera 52 a, 52 b, 52 c, 52 d may be coupled tothe network 104 to communicate data between the one or more of the 102a, 102 b, 102 c, 102 d, and the conference system 106.

Some examples of the IOT devices 52 a, 52 b, 52 c, 52 d that can includea video camera for include a personal computer, a computer monitor, aphone, a laptop, a tablet computer, a lightbulb, a luminaire, a lightingsystem, a door lock, a water heater, a sprinkler system, anair-conditioner, a thermostat, an alarm clock, a window shade, a switch,a smoke alarm, an electrical outlet, an alarm, a personal proximitysensor, a door sensor, a biometric sensor, a mobile device, anautomotive sensor, a cooking device, an electrical breaker, a personalalert sensor, a motion sensor, a calendar, a television, a radio, aradio frequency identification (RFID) tag/RFID detector, a vehicle, anelectric vehicle charger, a distributed generator (e.g. solar panel), adistributed energy storage (e.g. battery), a thermometer, andcombinations thereof.

The type of camera employed in the IOT devices 52 a, 52 b, 52 c, 52 dmay be any camera that can determine whether the user's attention is onthe multi-party conversation. For example, if the user is engaged inconversation in a person in his office while the multi-party phoneconversation is pending, the IOT devices 52 a, 52 b, 52 c, 52 d canrecord that activity and forward the information over the network 104 tothe conference system 106. The conference system 106 can then analyzethe video and determine if an alert need to be sent to the user 102 a,102 b, 102 c, 102 d to indicate to the user that the user's attentionshould be on the multi-party conversation based upon the user'sindividual status in the conversation.

The haptic feedback device 51 a, 51 b, 51 c, 51 d in one embodimentincludes an actuator, such as, e.g., an electric motor, anelectro-magnetic actuator, a voice coil, a linear resonant actuator, apiezoelectric actuator, a shape memory alloy, an electro-active polymer,a solenoid, an eccentric rotating mass motor (“ERM”) or a linearresonant actuator (“LRA”), a high bandwidth actuator, an electroactivepolymer (“EAP”) actuator, an electrostatic friction display, or anultrasonic vibration generator. In addition to an actuator, the hapticfeedback device 51 a, 51 b, 51 c, 51 d may be a non-mechanical ornon-vibratory device such as devices that use electrostatic friction(“ESF”), ultrasonic surface friction (“USF”), devices that induceacoustic radiation pressure with an ultrasonic haptic transducer,devices that use a haptic substrate and a flexible or deformable surfaceor shape changing devices and that may be attached to a user's body,devices that provide projected haptic output such as a puff of air usingan air jet, etc. Multiple haptic output devices with multiple hapticeffects can generate a haptic effect.

The generated haptic effects can include a wide range of effects andtechnologies, including vibrations, deformation, squeezing, poking,stretching, surface friction, heat, etc. A device that generates hapticeffects, and includes haptic feedback device 51 a, 51 b, 51 c, 51 d, canbe a wearable device (e.g., a bracelet, armband, glove, jacket, vest,pair of glasses, shoes, belt, etc.), a handheld device (e.g., a mobilephone, computer mouse, etc.), haptically enabled furniture (e.g., achair, couch, etc.) or any other haptically enabled device. In oneexample, the haptic feedback device 51 a, 51 b, 51 c, 51 d is asmartwatch. In another example, the haptic feedback device 51 a, 51 b,51 c, 51 d is a Fitbit™ activity tracker available from Fitbit Inc.

The haptic feedback device 51 a, 51 b, 51 c, 51 d can include cellularradio to establish a connection to the Internet via a cellular servicesuch as a 4G (e.g., Mobile WiMAX, LTE) or 5G cellular data service toaccess the network 104; or the haptic feedback device 51 a, 51 b, 51 c,51 d may include a WiFi radio to establish a WiFi connection through aWiFi access point or router which couples the haptic feedback device 51a, 51 b, 51 c, 51 d to the Internet (e.g., via an Internet ServiceProvider providing Internet service to the end user), which in turnconnect to the conference system 106/network 104. Of course, it shouldbe noted that the underlying principles of the invention are not limitedto any particular type of communication channel or protocol. Forexample, in one embodiment, the haptic feedback device 51 a, 51 b, 51 c,51 d may be equipped with Bluetooth LE radios and protocol stacks.

As noted above, the conference system 106 determining that a user 102 a,102 b, 102 c, 102 d is not directing his attention to the multi-partyconversation, or a user 102 a, 102 b, 102 c, 102 d that is attempting tocontribute to the multi-party conversation yet cannot because the user102 a, 102 b, 102 c, 102 d has set his communication device 53 a, 53 b,53 c, 53 d to mute; the conference system 106 can send a signal to theuser 102 a, 102 b, 102 c, 102 d via the haptic feedback device 51 a, 51b, 51 c, 51 d that he should pay attention to the multi partyconversation and/or un-mute his communication device 53 a, 53 b, 53 c,53 d.

One or more portions of the network 104 may be an ad hoc network, anintranet, an extranet, a virtual private network (VPN), a local areanetwork (LAN), a wireless LAN (WLAN), a wide area network (WAN), awireless WAN (WWAN), a metropolitan area network (MAN), a portion of theInternet, a portion of the Public Switched Telephone Network (PSTN), acellular telephone network, a wireless network, a WiFi network, a WiMaxnetwork, another type of network, or a combination of two or more suchnetworks. Any one or more portions of the network 104 may communicateinformation via a transmission medium. As used herein, “transmissionmedium” refers to any intangible (e.g., transitory) medium that iscapable of communicating (e.g., transmitting) instructions for executionby a machine (e.g., by one or more processors of such a machine), andincludes digital or analog communication signals or other intangiblemedia to facilitate communication of such software.

The conference system 106 manages a communication session (also referredto as a “conference call” or “call”) between the users 102 a, 102 b, 102c, 102 d. While a total of four users or participants are shown in theconference call illustrated in FIG. 1, any number of users may beinvolved in the conference call. The conference system 106 provides forrecording the data provided by the communication devices 53 a, 53 b, 53c, 53 d, and recording the data collected by the IOT devices 52 a, 52 b,52 c, 52 d. The conference system 106 analyzes the data, and determinesif the content of the communications session and the status of said atleast two users calls for input by the user through said user specificcommunications device.

Thereafter, the conference system 106 sends a feedback signal to theuser specific haptic feedback device 51 a, 51 b, 51 c, 51 d of the userthat the conference system 106 has determined that the communicationssession has called for the input by the user.

It is noted that the environment 500 shown in FIG. 1 is merely anexample. For instance, not all components of the environment 500 may beshown. Additionally, any two or more of the systems, devices, ormachines illustrated in FIG. 1 may be combined into a single system,device, or machine, and the functions described herein for any singlesystem or machine may be subdivided among multiple systems or machines.

Any of the systems or machines (e.g., devices) shown in FIG. 1 may be,include, or otherwise be implemented in a special-purpose (e.g.,specialized or otherwise non-generic) computer that has been modified(e.g., configured or programmed by software, such as one or moresoftware modules of an application, operating system, firmware,middleware, or other program) to perform one or more of the functionsdescribed herein for that system or machine. For example, aspecial-purpose computer system able to implement any one or more of themethodologies described herein is discussed below with respect to FIGS.2 and 3, and such a special-purpose computer may, accordingly, be ameans for performing any one or more of the methodologies discussedherein. Within the technical field of such special-purpose computers, aspecial-purpose computer that has been modified by the structuresdiscussed herein to perform the functions discussed herein istechnically improved compared to other special-purpose computers thatlack the structures discussed herein or are otherwise unable to performthe functions discussed herein. Accordingly, a special-purpose machineconfigured according to the systems and methods discussed hereinprovides an improvement to the technology of similar special-purposemachines.

FIG. 2 is a block/flow diagram showing a method of providing feedback toparticipants of multi-party phone conversations of their individualstatus in the conversation.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

In some embodiments, the method may begin at block 1, which can includeopening a communications session with a conference system 106 for atleast two users (102 a, 102 b, 102 c, 102 d), wherein each of the atleast two users has a user specific communications device 53 a, 53 b, 53c, 54 d registered with the conference system, a user specific conductmeasuring device, e.g., user specific IOT device 52 a, 52 b, 52 c, 52 d,registered with the conference system, and a user specific hapticfeedback device 51 a, 51 b, 51 c, 51 d registered with the conferencesystem. Registration means that addresses are provided to the conferencesystem 106 so that all the devices may be reached over the network 104.Registration for the user specific communications device 53 a, 53 b, 53c, 54 d may include the entry of phone numbers by the users into theconference system 106. This information may be stored in a registrationdatabase 20 that includes a plurality of user accounts, e.g., userspecific communications device database 21 (database of addresses forthe user specific communications device 53 a, 53 b, 53 c, 54 d), userspecific conduct measuring device database 22 (database of addresses forthe user specific TOT devices 52 a, 52 b, 52 c, 52 d), and a userspecific haptic feedback device database 23 (database of addresses forthe user specific haptic feedback device 51 a, 51 b, 51 c, 51 d). Eachof these databases can include a form of hardware memory for storingaddress data.

It is noted that the term “user specific” is meant to denote that eachuser has their own communications device having an address that isunique to the user, each user has their own conduct measuring devicehaving an address that is unique to the user, and each user has theirown haptic feedback device having an address that is unique to the user.

Opening a communications session may involve one of the users 102 a, 102b, 102 c, 102 d functioning as a “host” to set up a communicationssession with the conference system 100. This can include logging intothe conference system 100 by providing identification that can identifyeach of the users.

The conference system 100 can provide a conference time and conferencelog in information, which can be distributed to all the users 102 a, 102b, 102 c, 102 d. It is not necessary that all users 102 a, 102 b, 102 c,102 d have conference time reserved for the communications sessionbefore the communications session begins. For example, a user 102 a, 102b, 102 c, 102 d can call into the communications session after it hasstarted and add all the required address information, e.g., addressesfor the user specific communications device 53 a, 53 b, 53 c, 54 d,addresses for the user specific IOT devices 52 a, 52 b, 52 c, 52 d (alsoreferred to as specific conduct measuring devices), and addresses forthe user specific haptic feedback device 51 a, 51 b, 51 c, 51 d), withthe conference system 100 to participate in the communications session.In other examples, a user may already have all their information savedby the conference system 100 from a prior interaction, and the call intothe system may automatically interconnect all of the user specificdevices with the call.

In some embodiments, the user specific communications device 53 a, 53 b,53 c, 54 d is a telephone with a mute function. By “mute function” it ismeant, that the communications device, e.g., telephone, may have theability to selective disable the voice receiving function, e.g.,microphone, so that any sound directed at the telephone is nottransmitted into a conversation, communication and/or conference beingconducted over a network of communication devices, such as othertelephones. It is noted that the mute function is reversible. Forexample, when the mute function is “ON”, sound directed at that deviceis not transmitted, however when the mute function is “OFF”, sounddirected at the device, e.g., telephone device, is transmitted into theconversation, communication and/or conference being conducted over thenetwork of communication devices, such as other telephones.

In some examples, the user specific conduct measuring device is aninternet of things (TOT) device 52 a, 52 b, 52 c, 52 d including atleast one of a camera and a microphone. In one example, the userspecific conduct measuring device is a voice assistant, which can listento the users statements being made during the time of the communicationssession. The voice assistance may also have video capabilities fordetermining the physical behavior of the user 102 a, 102 b, 102 c, 102 dduring the time of the communications session. The user specificconductive measuring device may be provided by more than one device. Forexample, recording of audio data from the users 102 a, 102 b, 102 c, 102d can be achieved using a voice assistance, while recording of videodata can be provided by IOT type security cameras, etc. In yet anotherexample, the microphone and video components that are built into a PCcomputer, e.g., the monitor of the PC computer, can also provide theaudio and video components for the user specific conduct measuringdevice. This is particularly useful for an office or cubicle typesetting.

The user specific haptic feedback device 51 a, 51 b, 51 c, 51 d assertsa signal by physical interaction with the user by an electric motor, anelectro-magnetic actuator, a linear resonant actuator, a piezoelectricactuator, a shape memory alloy, an electro-active polymer, a solenoid,an eccentric rotating mass motor (“ERM”), a linear resonant actuator(“LRA”), a high bandwidth actuator, an electroactive polymer (“EAP”)actuator, an electrostatic friction display, an ultrasonic vibrationgenerator, an electrostatic friction (“ESF”) device, and ultrasonicsurface friction (“USF”) or a combination thereof. In one embodiment,the specific haptic feedback device 51 a, 51 b, 51 c, 51 d has a formfactor to be worn on a wrist of the user.

It is also noted that embodiments have been contemplated in which theuser specific IOT devices 52 a, 52 b, 52 c, 52 d (also referred to asspecific conduct measuring devices) are not provided by the usersthemselves, but happen to in an area, in which the users are presentduring the time of the communications session 100. In this example, theconference system 100 may be able to match user specific IOT devices 52a, 52 b, 52 c, 52 d with users 102 a, 102 b, 102 c, 102 d by the usersproviding their location, or employing GPS access to determine thepresence of IOT devices for determining the conduct of the users.

Referring to FIG. 2, at block 2, the method can continue with analyzingcontent of the communications session from content received by theconference system 100 through the user specific communications device 53a, 53 b, 53 c, 53 d for at least one of the users. Analyzing the contentof the communications session may include recording the content of themulti-party discussion that results from at least one or two of themultiple users 102 a, 102 b, 102 c, 102 d speaking into their ownspecific communications device 53 a, 53 b, 53 c, 53 d. The recording andstorage of that information may be provided by the content inputrecorder 25 of the conference system 100 depicted in FIG. 3. The contentinput recorder 25 may include any type of memory and/or media capable ofrecording voice transmissions.

In some embodiments, the analyzing of the content of the communicationssession from the content received by the conference system 100 throughthe user specific communications device 53 a, 53 b, 53 c, 53 d for theat least one of the users 102 a, 102 b, 102 c, 102 d can includecapturing a phone conversation for all participants from the userspecific communication device 53 a, 53 b, 53 c, 53 d; and measuring avolume for the participants from the user specific communication device.In some instances, measuring of the volume includes determining for theparticipants whether user specific communication device 53 a, 53 b, 53c, 53 d for at least one of the users is on mute. The analysis caninclude determining the content of the conversation by differentsubjects. The analysis can further include a feature for determiningwhen a specific user, e.g., by user name, is being called forinformation in the communications session.

Referring to FIG. 2, at block 3, the method can continue with capturingstatus for said at least two users 102 a, 102 b, 102 c, 102 d from datameasured by the user specific conduct measuring device, e.g., internetof things (JOT) device 52 a, 52 b, 52 c, 52 d, including at least one ofa camera and a microphone, for the at least two users. In someembodiments, capturing status for said at least two users from datameasured by the user specific conduct measuring device for the at leasttwo users includes determining with the camera of the user specificconduct measuring device 53 a, 53 b, 53 c, 53 d a direction of the users102 a, 102 b, 102 c, 102 d while talking with respect to the userspecific communication device 53 a, 53 b, 53 c, 53 d. Determining thedirection of a user 102 a, 102 b, 102 c, 102 d can be used to determinewhether a user 102 a, 102 b, 102 c, 102 d is paying attention to thecontent of the communications session, or whether a user 102 a, 102 b,102 c, 102 d is not paying attention to the content of thecommunications session. For example, if the user 102 a, 102 b, 102 c,102 d is facing the user specific communication device 53 a, 53 b, 53 c,53 d, which may be a telephone or a computer including a microphone, itcan be assumed that the user 102 a, 102 b, 102 c, 102 d is an activeparticipant in the communications session, i.e., the user 102 a, 102 b,102 c, 102 d is paying attention. For example, if the user 102 a, 102 b,102 c, 102 d is facing away from the user specific communication device53 a, 53 b, 53 c, 53 d, which may be a telephone or a computer includinga microphone, it can be assumed that the user 102 a, 102 b, 102 c, 102 dis not an active participant in the communications session, i.e., theuser 102 a, 102 b, 102 c, 102 d is not paying attention. The directionof the users 102 a, 102 b, 102 c, 102 d can be measured using a videocamera, e.g., the video camera of the user specific conduct measuringdevice, e.g., internet of things (TOT) device 52 a, 52 b, 52 c, 52 d.

In another example, determining a distance of users 102 a, 102 b, 102 c,102 d while talking with respect to the user specific communicationdevice 53 a, 53 b, 53 c, 53 d can be used to determine whether a user102 a, 102 b, 102 c, 102 d is paying attention to the content of thecommunications session, or whether a user 102 a, 102 b, 102 c, 102 d isnot paying attention to the content of the communications session. Forexample, if the user 102 a, 102 b, 102 c, 102 d is within arm's lengthof the user specific communication device 53 a, 53 b, 53 c, 53 d, whichmay be a telephone or a computer including a microphone, it can beassumed that the user 102 a, 102 b, 102 c, 102 d is an activeparticipant in the communications session, i.e., the user 102 a, 102 b,102 c, 102 d is paying attention. For example, if the user 102 a, 102 b,102 c, 102 d is far away from the user specific communication device 53a, 53 b, 53 c, 53 d, which may be a telephone or a computer including amicrophone, which can mean more than arm's length from the user specificcommunication device 53 a, 53 b, 53 c, 53 d, it can be assumed that theuser 102 a, 102 b, 102 c, 102 d is not an active participant in thecommunications session, i.e., the user 102 a, 102 b, 102 c, 102 d is notpaying attention. The distance of the users 102 a, 102 b, 102 c, 102 dfrom the user specific communication device 53 a, 53 b, 53 c, 53 d canbe measured using a video camera, e.g., the video camera of the userspecific conduct measuring device, e.g., internet of things (TOT) device52 a, 52 b, 52 c, 52 d.

In yet another example, determining whether a user 102 a, 102 b, 102 c,102 d is speaking with another party directly while the multi-partyphone conversation is pending can be used to determine whether a user102 a, 102 b, 102 c, 102 d is paying attention to the content of thecommunications session, or whether a user 102 a, 102 b, 102 c, 102 d isnot paying attention to the content of the communications session. Forexample, if the user 102 a, 102 b, 102 c, 102 d is not speaking with athird party that is not a party to the multi-party conversation of thecommunications session it can be assumed that the user 102 a, 102 b, 102c, 102 d is an active participant in the communications session, i.e.,the user 102 a, 102 b, 102 c, 102 d is paying attention. For example, ifthe user 102 a, 102 b, 102 c, 102 d is speaking with a third party thatis not a party to the multi-party conversation of the communicationssession it can be assumed that the user 102 a, 102 b, 102 c, 102 d isnot an active participant in the communications session, i.e., the user102 a, 102 b, 102 c, 102 d is not paying attention. The presence ofthird parties speaking to the users 102 a, 102 b, 102 c, 102 d can bedetected using a video camera, e.g., the video camera of the userspecific conduct measuring device, e.g., internet of things (TOT) device52 a, 52 b, 52 c, 52 d, as well as an audio source. Facial recognitionmay be employed. Additionally, the conversation with a third party canbe analyzed to determine is subject and applicability to the subject ofthe multi-party conversation of the communications session. It is notnecessary that the third party be a person, or that the noise made bythe third party be a voice communication. In some instances, the thirdparty can be a pet, such as a dog or cat. In this instance, the noisecollected by the user specific conduct measuring device, e.g., internetof things (TOT) device 52 a, 52 b, 52 c, 52 d, may be pet emitted sound,such as a bark or cat. The conference system 100 could flag these soundsas being inappropriate to the subject of the multi-party conversation,especially if also recorded over the user specific communication device53 a, 53 b, 53 c, 53 d so that all users 102 a, 102 b, 102 c, 102 d canhere the irrelevant noise. The irrelevant background noise does not haveto be limited to pet noises. Any background noise may be analyzed. Forexample, third party conversations with people not party to thecommunications session, TV, radio, intercom announcements, buildingnoise, as well as other background noises can be measured specific to auser, analyzed by the conference system 100, and stopped from furthertransmission by sending a signal from the conference system 100 to theuser 102 a, 102 b, 102 c, 102 d to mute his particular communicationdevice 53 a, 53 b, 53 c, 53 d.

Referring to FIG. 3, the recording and storage of the conduct of users102 a, 102 b, 102 c, 102 d measured by the user specific conductmeasuring device, e.g., internet of things (JOT) device 52 a, 52 b, 52c, 52 d, may be provided by the conduct input recorder 30 of theconference system 100 depicted in FIG. 3. The conduct input recorder 30may include any type of memory and/or media capable of recording videoand/or audio transmissions.

Referring to FIG. 2, at block 4, the method can continue withdetermining with the conference system 100 if the content of thecommunications session and the status of at least two users 102 a, 102b, 102 c, 102 d calls for input by the user 102 a, 102 b, 102 c, 102 dthrough the user specific communications device 53 a, 53 b, 53 c, 53 d.An “input” by a user 102 a, 102 b, 102 c, 102 d can be that during themulti-party conversation of the communications session content is raisedby at least one of the users that a second user is to address. Thecontent could designate a party, i.e., user 102 a, 102 b, 102 c, 102 d,by name. For example, one user may specifically address or request ananswer, i.e., a call for input to the conversation, from a second user,e.g., by name, or title. The content of the communications session coulddesignate a party, i.e., user 102 a, 102 b, 102 c, 102 d, by subject ofthe communication, or a change in subject of the communication. Forexample, if the subject of the multi-party conversation of thecommunication session was to change from discussing transportation to ameeting to security at a meeting, the subject of the communication canchange to security, and all users having security responsibility may becalled for input to the conversation.

In some embodiments, determining with the conference system 100 if thecontent of the communications session and the status of at least twousers 102 a, 102 b, 102 c, 102 d calls for input by the user through auser specific communications device 53 a, 53 b, 53 c, 53 d includesdetermining that the user 102 a, 102 b, 102 c, 102 d is attempting tocontribute to the multi-party phone conversation by speaking into theuser specific communication device 53 a, 53 b, 53 c, 53 d while it isset on mute. This can be determined at block 2 of the method, in whichduring that portion of the sequence conversation from the users 102 a,102 b, 102 c, 102 d is recorded through the user specific communicationdevice 53 a, 53 b, 53 c, 53 d, and the volume of the transmission ismeasured, e.g., whether the user specific communication device 53 a, 53b, 53 c, 53 d, is on mute setting. Once a mute setting is determined, itcan be further determined from the user specific conduct measuringdevices, e.g., internet of things (IOT) devices 52 a, 52 b, 52 c, 52 d,whether a user is trying to introduce voice content/message to themulti-party conversation in the communication session over a userspecific communication device 53 a, 53 b, 53 c, 53 d that is set on muteusing the video and/or audio capabilities of the user specific conductmeasuring devices, e.g., internet of things (IOT) devices 52 a, 52 b, 52c, 52 d. If a user is trying to communicate over a phone/device set tomute, the conference system 100 calls for an input to the user to turnthe mute setting of the phone/device off.

In some embodiments, determining with the conference system 100 if thecontent of the communications session and the status of at least twousers 102 a, 102 b, 102 c, 102 d calls for input by the user 102 a, 102b, 102 c, 102 d through user specific communications device includesdetermining that the communications session includes the name of anindividual user, determining the individual user is not facing userspecific communications device 53 a, 53 b, 53 c, 53 d with the userspecific conduct measuring device, and issuing a call for input by theuser 102 a, 102 b, 102 c, 102 d. In this instance, the content of themulti-party conversation has directed to a person, i.e., user, that isnot paying attention to the conversation. The call for input is to getthe user 102 a, 102 b, 102 c, 102 d to redirect his attention back tothe conversation. Additionally, it is not required that the conversationspecifically designate a user by name in this example. The multi-partyconversation may designate a user 102 a, 102 b, 102 c, 102 d by asubject of the conversation, in which the subject is correlated with aspecific user. Additionally, the location, e.g., distance of the user102 a, 102 b, 102 c, 102 d, may be used a substitution or in combinationwith the direction of the user 102 a, 102 b, 102 c, 102 d in determininga user's degree of attention.

In another embodiment, determining with the conference system 100 if thecontent of the communications session and the status of at least twousers calls 102 a, 102 b, 102 c, 102 d for input by the user throughuser specific communications device includes determining that thecommunications session includes background noise not relevant to themulti-party conversation being measured from the user specificcommunications device, determining the specific communications device isnot muted; and issuing calls for input to the user corresponding to thespecific communications device. If a user is trying to communicate overa phone/device set to mute, the conference system 100 calls for an inputto the user to turn the mute setting of the phone/device off.

The step of determining with conference system 100 if the content of thecommunications session and the status of at least two users calls 102 a,102 b, 102 c, 102 d for input by the user through user specificcommunications device may be provided by an individual status calculator35 that is depicted in FIG. 3. The individual status calculator 35 canat least one hardware device processor for performing a set ofinstruction stored on at least one memory device, in which theindividual status calculator 35 analyzes the content of thecommunications session received from the user specific communicationsdevice from at least two users that is recorded and stored in thecontent input recorder 25, analyzing the status of said at least twousers measured by the user specific conduct measuring device signalsrecorded and stored in the conductive input recorder 30, and calculatingwhether an individual status for one of said at least two users callsfor an input through said user specific communications device by theuser having the individual status calling for the input. Calculatingwhether an individual status for one of said at least two users callsfor an input may include cognitive analysis of inputs from theconductive input recorder 30 and the content input recorder 25 todetermine state conditions, providing a weight for each condition, andcompare defined actions to weighted result of all inputs. The result ofthe inputs illustrates when a call for an input from a user is needed.

The individual status calculator 35 may be provided by some form orartificial intelligence providing device, such as an artificial neuralnetwork providing device. An artificial neural network (ANN) is aninformation processing system that is inspired by biological nervoussystems, such as the brain. The key element of ANNs is the structure ofthe information processing system, which includes a large number ofhighly interconnected processing elements (called “neurons”) working inparallel to solve specific problems. ANNs are furthermore trainedin-use, with learning that involves adjustments to weights that existbetween the neurons. An ANN is configured for a specific application,such as pattern recognition or data classification, through such alearning process.

Referring now to FIG. 5, a generalized diagram of a neural network isshown. ANNs demonstrate an ability to derive meaning from complicated orimprecise data and can be used to extract patterns and detect trendsthat are too complex to be detected by humans or other computer-basedsystems. The structure of a neural network is known generally to haveinput neurons 103 that provide information to one or more “hidden”neurons 107. Connections between the input neurons 103 and hiddenneurons 107 are weighted and these weighted inputs are then processed bythe hidden neurons 107 according to some function in the hidden neurons107, with weighted connections between the layers. There may be anynumber of layers of hidden neurons 107, and as well as neurons thatperform different functions. There exist different neural networkstructures as well, such as convolutional neural network, maxoutnetwork, etc. Finally, a set of output neurons 109 accepts and processesweighted input from the last set of hidden neurons 107.

This represents a “feed-forward” computation, where informationpropagates from input neurons 103 to the output neurons 109. Uponcompletion of a feed-forward computation, the output is compared to adesired output available from training data. The error relative to thetraining data is then processed in “feed-back” computation, where thehidden neurons 107 and input neurons 103 receive information regardingthe error propagating backward from the output neurons 109. Once thebackward error propagation has been completed, weight updates areperformed, with the weighted connections being updated to account forthe received error. This represents just one variety of ANN. Thedescription of the functioning of the neural network provided withreference to FIG. 5 can provide one example of how the data fromcollected from blocks 2 and 3 of the method depicted in FIG. 2 canprovide the data for determining with the individual status calculator35 depicted in FIG. 3 whether the content of the communications sessionand the status of at least two users calls for input by the user.

FIG. 6 illustrates an example of an artificial neural network (ANN)architecture 200. It should be understood that the present architectureis purely exemplary and that other architectures or types of neuralnetwork may be used instead. During feed-forward operation, a set ofinput neurons 202 each provide an input voltage in parallel to arespective row of weights 204. The weights 204 each have a settableresistance value, such that a current output flows from the weight 204to a respective hidden neuron 206 to represent the weighted input. Thecurrent output by a given weight is determined as I=v/r, where V is theinput voltage from the input neuron 202 and r is the set resistance ofthe weight 204. The current from each weight adds column-wise and flowsto a hidden neuron 206. A set of reference weights 207 have a fixedresistance and combine their outputs into a reference current that isprovided to each of the hidden neurons 206. Because conductance valuescan only be positive numbers, some reference conductance is needed toencode both positive and negative values in the matrix. The currentsproduced by the weights 204 are continuously valued and positive, andtherefore the reference weights 207 are used to provide a referencecurrent, above which currents are considered to have positive values andbelow which currents are considered to have negative values.

As an alternative to using the reference weights 207, another embodimentmay use separate arrays of weights 204 to capture negative values. Eachapproach has advantages and disadvantages. Using the reference weights207 is more efficient in chip area, but reference values need to bematched closely to one another. In contrast, the use of a separate arrayfor negative values does not involve close matching as each value has apair of weights to compare against. However, the negative weight matrixapproach uses roughly twice the chip area as compared to the singlereference weight column. In addition, the reference weight columngenerates a current that needs to be copied to each neuron forcomparison, whereas a negative matrix array provides a reference valuedirectly for each neuron. In the negative array embodiment, the weights204 of both positive and negative arrays are updated, but this alsoincreases signal-to-noise ratio as each weight value is a difference oftwo conductance values. The two embodiments provide identicalfunctionality in encoding a negative value and those having ordinaryskill in the art will be able to choose a suitable embodiment for theapplication at hand.

The hidden neurons 206 use the currents from the array of weights 204and the reference weights 207 to perform some calculation. The hiddenneurons 206 then output a voltage of their own to another array ofweights 204. This array performs in the same way, with a column ofweights 204 receiving a voltage from their respective hidden neuron 206to produce a weighted current output that adds row-wise and is providedto the output neuron 208.

It should be understood that any number of these stages may beimplemented, by interposing additional layers of arrays and hiddenneurons 206. It should also be noted that some neurons may be constantneurons 209, which provide a constant voltage to the array. The constantneurons 209 can be present among the input neurons 202 and/or hiddenneurons 206 and are only used during feed-forward operation.

During back propagation, the output neurons 208 provide a voltage backacross the array of weights 204. The output layer compares the generatednetwork response to training data and computes an error. The error isapplied to the array as a voltage pulse, where the height and/orduration of the pulse is modulated proportional to the error value. Inthis example, a row of weights 204 receives a voltage from a respectiveoutput neuron 208 in parallel and converts that voltage into a currentwhich adds column-wise to provide an input to hidden neurons 206. Thehidden neurons 206 combine the weighted feedback signal with aderivative of its feed-forward calculation and stores an error valuebefore outputting a feedback signal voltage to its respective column ofweights 204. This back propagation travels through the entire network200 until all hidden neurons 206 and the input neurons 202 have storedan error value.

During weight updates, the input neurons 202 and hidden neurons 206apply a first weight update voltage forward and the output neurons 208and hidden neurons 206 apply a second weight update voltage backwardthrough the network 200. The combinations of these voltages create astate change within each weight 204, causing the weight 204 to take on anew resistance value. In this manner the weights 204 can be trained toadapt the neural network 200 to errors in its processing. It should benoted that the three modes of operation, feed forward, back propagation,and weight update, do not overlap with one another.

Referring now to FIG. 7, a block diagram of a neuron 300 is shown. Thisneuron may represent any of the input neurons 202, the hidden neurons206, or the output neurons 208. It should be noted that FIG. 7 showscomponents to address all three phases of operation: feed forward, backpropagation, and weight update. However, because the different phases donot overlap, there will necessarily be some form of control mechanismwithin in the neuron 300 to control which components are active. Itshould therefore be understood that there may be switches and otherstructures that are not shown in the neuron 300 to handle switchingbetween modes.

In feed forward mode, a difference block 302 determines the value of theinput from the array by comparing it to the reference input. This setsboth a magnitude and a sign (e.g., + or −) of the input to the neuron300 from the array. Block 304 performs a computation based on the input,the output of which is stored in storage 305. It is specificallycontemplated that block 304 computes a non-linear function and may beimplemented as analog or digital circuitry or may be performed insoftware. The value determined by the function block 304 is converted toa voltage at feed forward generator 306, which applies the voltage tothe next array. The signal propagates this way by passing throughmultiple layers of arrays and neurons until it reaches the final outputlayer of neurons. The input is also applied to a derivative of thenon-linear function in block 308, the output of which is stored inmemory 309.

During back propagation mode, an error signal is generated. The errorsignal may be generated at an output neuron 208 or may be computed by aseparate unit that accepts inputs from the output neurons 208 andcompares the output to a correct output based on the training data.Otherwise, if the neuron 300 is a hidden neuron 206, it receives backpropagating information from the array of weights 204 and compares thereceived information with the reference signal at difference block 310to provide a continuously valued, signed error signal. This error signalis multiplied by the derivative of the non-linear function from theprevious feed forward step stored in memory 309 using a multiplier 312,with the result being stored in the storage 313. The value determined bythe multiplier 312 is converted to a backwards propagating voltage pulseproportional to the computed error at back propagation generator 314,which applies the voltage to the previous array. The error signalpropagates in this way by passing through multiple layers of arrays andneurons until it reaches the input layer of neurons 202.

During weight update mode, after both forward and backward passes arecompleted, each weight 204 is updated proportional to the product of thesignal passed through the weight during the forward and backward passes.The update signal generators 316 provide voltage pulses in bothdirections (though note that, for input and output neurons, only onedirection will be available). The shapes and amplitudes of the pulsesfrom update generators 316 are configured to change a state of theweights 204, such that the resistance of the weights 204 is updated.

In one particular embodiment, the weights 204 may be implemented insoftware or in hardware, for example using relatively complicatedweighting circuitry or using resistive cross point devices. Suchresistive devices may have switching characteristics that have anon-linearity that can be used for processing data. The weights 204 maybelong to a class of device called a resistive processing unit (RPU),because their non-linear characteristics are used to performcalculations in the neural network 200. The RPU devices may beimplemented with resistive random access memory (RRAM), phase changememory (PCM), programmable metallization cell (PMC) memory, or any otherdevice that has non-linear resistive switching characteristics. Such RPUdevices may also be considered as memristive systems.

The artificial neural networks described above with reference to FIGS.5-7 provide one mechanism of how the data from collected from blocks 2and 3 of the method depicted in FIG. 2 can provide the data fordetermining with the individual status calculator 35 depicted in FIG. 3whether the content of the communications session and the status of atleast two users calls for input by the user.

Referring to FIG. 2, at block 5, the method can continue with sending afeedback signal from the conference system 100 to the user specifichaptic feedback device 51 a, 51 b, 51 c, 51 d of the user 102 a, 102 b,102 c, 102 s that the conference system has determined that thecommunications session has called for the input. The user specifichaptic feedback device 51 a, 51 b, 51 c, 51 d then delivers the feedbacksignal to the user. The user is then alerted that a mute function of hiscommunication device needs to be adjusted and/or his attention is to beredirected to the multi-person conversation of the communicationssession.

In one example, a user 102 a, 102 b, 102 c, 102 d is on a phone call andhas placed his phone on mute. He starts to respond. The system 100vibrates his smartwatch, i.e., haptic feedback device 51 a, 51 b, 51 c,51 d, for a time period of 3 seconds based on his configuration. Thehaptic signal illustrates to the user that the mute function of hisphone is to be turned off

In another example, a user 102 a, 102 b, 102 c, 102 d is on a phone calland has placed his phone on mute. He starts talking to his father, i.e.,third party that is not a part of the multi-party conversation of thecommunication across the room. Based on IoT sensors determining he isnot talking towards his phone and other conversation going on, nofeedback is initiated.

In yet another example, while the user 102 a, 102 b, 102 c, 102 d is onthe phone call, the user's name is mentioned in the multi-personconversation. The system determines the user is not engaged in theconversation. The system 100 pulses that users fitbit, i.e., hapticfeedback device 51 a, 51 b, 51 c, 51 d, for a period of 3 times. Theuser is alerted that her attention is needed for the conversation.

In a further example, while the user 102 a, 102 b, 102 c, 102 d is onthe phone call, the user's name is mentioned in the multi-personconversation. The system determines the user is engaged in theconversation. The system does not send a signal to the user's hapticfeedback device 51 a, 51 b, 51 c, 51 d.

In an even further example, while the user 102 a, 102 b, 102 c, 102 d ison the phone call, the user's name is mentioned in the multi-personconversation. The system determines the user is not engaged in theconversation. The system 100 pulses that users fitbit, i.e., hapticfeedback device 51 a, 51 b, 51 c, 51 d, for a period of 3 times. Basedon the lack of response, the strength and frequency of the feedbackincreases until he responds. The user is alerted that her attention isneeded for the conversation.

FIG. 3 is a block diagram illustrating a conferencing system 100 forproviding feedback to participants of multi-party phone conversations oftheir individual status in the conversation, in accordance with oneembodiment of the present disclosure. The conferencing system 100 may beused in combination with the method described in FIG. 2. Some elementsof the conferencing system 100 have already been discussed in thedescription of FIG. 2.

In some embodiments, the conferencing system 100 that providing feedbackto participants of multi-party phone conversations includes aregistration database 20 that includes a plurality of user accountsincluding addresses for user specific communications device 21registered with the conference system, a user specific conduct measuringdevices 22, and a user specific haptic feedback devices 23. Theconferencing system 100 may also include a content input recorder 25 forreceiving communications content from a communications session fromcontent received through the user specific communications device for atleast two of the users in the registration database. The conferencingsystem 100 may also include a conduct input recorder 25 for recording acapturing status for said at least two users from data measured by theuser specific conduct measuring device.

In some embodiments, the conferencing system 100 includes an individualstatus calculator 35 including at least one hardware device processor 40for performing a set of instruction stored on at least one memory device15. The individual status calculator 35 can analyze the content of thecommunications session received from the user specific communicationsdevice 53 a, 53 b, 53 c, 53 d from at least two users, analyze thestatus of said at least two users measured by the user specific conductmeasuring device 52 a, 52 b, 52 c, 52 d, and calculate whether anindividual status for one of said at least two users calls for an inputthrough said user specific communications device 53 a, 53 b, 53 c, 53 dby the user having the individual status calling for the input.

In some embodiments, the conferencing system 100 also includes afeedback signal transmitter 36 that sends a signal for input from theuser. The signal is sent to the user's specific haptic feedback device51 a, 51 b, 51 c, 51 d. The user specific haptic feedback device 51 a,51 b, 51 c, 51 d delivers the feedback signal to the user by thephysical interaction provided by the user specific haptic feedbackdevice 51 a, 51 b, 51 c, 51 d. It is noted that each of theaforementioned elements of the conferencing system 100 are operativelycoupled via a system bus 102.

The conferencing system 100 may be integrated into the processing system400 depicted in FIG. 4. The processing system 400 includes at least oneprocessor (CPU) 104 operatively coupled to other components via a systembus 102. A cache 106, a Read Only Memory (ROM) 108, a Random AccessMemory (RAM) 110, an input/output (I/O) adapter 120, a sound adapter130, a network adapter 140, a user interface adapter 150, and a displayadapter 160, are operatively coupled to the system bus 102. The bus 102interconnects a plurality of components has will be described herein.

The system 400 depicted in FIG. 4, may further include a first storagedevice 122 and a second storage device 124 are operatively coupled tosystem bus 102 by the I/O adapter 120. The storage devices 122 and 124can be any of a disk storage device (e.g., a magnetic or optical diskstorage device), a solid state magnetic device, and so forth. Thestorage devices 122 and 124 can be the same type of storage device ordifferent types of storage devices.

A speaker 132 is operatively coupled to system bus 102 by the soundadapter 130. A transceiver 142 is operatively coupled to system bus 102by network adapter 140. A display device 162 is operatively coupled tosystem bus 102 by display adapter 160.

A first user input device 152, a second user input device 154, and athird user input device 156 are operatively coupled to system bus 102 byuser interface adapter 150. The user input devices 152, 154, and 156 canbe any of a keyboard, a mouse, a keypad, an image capture device, amotion sensing device, a microphone, a device incorporating thefunctionality of at least two of the preceding devices, and so forth. Ofcourse, other types of input devices can also be used, while maintainingthe spirit of the present invention. The user input devices 152, 154,and 156 can be the same type of user input device or different types ofuser input devices. The user input devices 152, 154, and 156 are used toinput and output information to and from system 400.

Of course, the processing system 400 may also include other elements(not shown), as readily contemplated by one of skill in the art, as wellas omit certain elements. For example, various other input devicesand/or output devices can be included in processing system 400,depending upon the particular implementation of the same, as readilyunderstood by one of ordinary skill in the art. For example, varioustypes of wireless and/or wired input and/or output devices can be used.Moreover, additional processors, controllers, memories, and so forth, invarious configurations can also be utilized as readily appreciated byone of ordinary skill in the art. These and other variations of theprocessing system 400 are readily contemplated by one of ordinary skillin the art given the teachings of the present invention provided herein.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

For example, the present disclosure provides a computer program productcomprising a non-transitory computer readable storage medium havingcomputer readable program code embodied therein for providing feedbackto participants of multi-party phone conversations. The method actuatedby the computer program product may include providing haptic feedback toparticipants of multi-party phone conversations that includes opening acommunications session with a conference system for at least two users,wherein each of the at least two users has a user specificcommunications device registered with the conference system, a userspecific conduct measuring device registered with the conference system,and a user specific haptic feedback device registered with theconference system. Analyzing content of the communications session fromcontent received by the conference system through the user specificcommunications device for at least one of the users; and capturingstatus for said at least two users from data measured by the userspecific conduct measuring device for the at least two users. The methodmay further include determining with the conference system if thecontent of the communications session and the status of said at leasttwo users calls for input by the user through said user specificcommunications device. A feedback signal is sent from the conferencesystem to the user specific haptic feedback device of the user that theconference system has determined that the communications session hascalled for said input. The user specific haptic feedback devicedeliverers the feedback signal to the user.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as SMALLTALK, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The methods of the present disclosure may be practiced using a cloudcomputing environment. Cloud computing is a model of service deliveryfor enabling convenient, on-demand network access to a shared pool ofconfigurable computing resources (e.g. networks, network bandwidth,servers, processing, memory, storage, applications, virtual machines,and services) that can be rapidly provisioned and released with minimalmanagement effort or interaction with a provider of the service. Thiscloud model may include at least five characteristics, at least threeservice models, and at least four deployment models. Characteristics areas follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based email). Theconsumer does not manage or control the underlying cloud infrastructureincluding network, servers, operating systems, storage, or evenindividual application capabilities, with the possible exception oflimited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting for loadbalancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 8, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 includes one or morecloud computing nodes 51 with which local computing devices used bycloud consumers, such as, for example, mobile and/or wearable electronicdevices 54A, desktop computer 54B, laptop computer 54C, and/orautomobile computer system 54N may communicate. Nodes 110 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 8 are intended to be illustrative only and that computing nodes51 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 9, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 8) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 9 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may include applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and application for the conferencing system100, which is described with reference to FIGS. 1-9.

Reference in the specification to “one embodiment” or “an embodiment” ofthe present invention, as well as other variations thereof, means that aparticular feature, structure, characteristic, and so forth described inconnection with the embodiment is included in at least one embodiment ofthe present invention. Thus, the appearances of the phrase “in oneembodiment” or “in an embodiment”, as well any other variations,appearing in various places throughout the specification are notnecessarily all referring to the same embodiment.

It is to be appreciated that the use of any of the following “/”,“and/or”, and “at least one of”, for example, in the cases of “A/B”, “Aand/or B” and “at least one of A and B”, is intended to encompass theselection of the first listed option (A) only, or the selection of thesecond listed option (B) only, or the selection of both options (A andB). As a further example, in the cases of “A, B, and/or C” and “at leastone of A, B, and C”, such phrasing is intended to encompass theselection of the first listed option (A) only, or the selection of thesecond listed option (B) only, or the selection of the third listedoption (C) only, or the selection of the first and the second listedoptions (A and B) only, or the selection of the first and third listedoptions (A and C) only, or the selection of the second and third listedoptions (B and C) only, or the selection of all three options (A and Band C). This may be extended, as readily apparent by one of ordinaryskill in this and related arts, for as many items listed.

Having described preferred embodiments of a haptic feedback to modifybehavior during phone calls (which are intended to be illustrative andnot limiting), it is noted that modifications and variations can be madeby persons skilled in the art in light of the above teachings. It istherefore to be understood that changes may be made in the particularembodiments disclosed which are within the scope of the invention asoutlined by the appended claims. Having thus described aspects of theinvention, with the details and particularity required by the patentlaws, what is claimed and desired protected by Letters Patent is setforth in the appended claims.

What is claimed is:
 1. A computer implemented method for providinghaptic feedback to participants of multi-party phone conversationscomprising: opening a communications session with a conference systemfor a conversation between at least two users, wherein each of the atleast two users has a user specific communications device registeredwith the conference system over which the conversation is beingtransmitted, a user specific conduct measuring device registered withthe conference system, and a user specific haptic feedback deviceregistered with the conference system; analyzing content of theconversation of the communications session between the at least twousers from content received by the conference system through the userspecific communications device for at least one of the users; capturingstatus for said at least two users from data measured by the userspecific conduct measuring device for the at least two users, the statusindicating the level of involvement of the at least two users in theconversation; determining with the conference system if the content ofthe communications session and the status of said at least two userscalls for input by the user to the conversation through said userspecific communications device; and sending a feedback signal from theconference system to the user specific haptic feedback device of theuser that the conference system has determined that the communicationssession has called for said input, the user specific haptic feedbackdevice delivering the feedback signal to the user.
 2. The computerimplemented method of claim 1, wherein the specific communicationsdevice includes a telephone with a mute function.
 3. The computerimplemented method of claim 1, wherein the specific conduct measuringdevice includes an internet of things (IOT) device including at leastone of a camera and a microphone.
 4. The computer implemented method ofclaim 1, wherein the specific conduct measuring device includes a voiceassistant.
 5. The computer implemented method of claim 1, wherein theuser specific haptic feedback device asserts a signal by physicalinteraction with the user by an electric motor, an electro-magneticactuator, a linear resonant actuator, a piezoelectric actuator, a shapememory alloy, an electro-active polymer, a solenoid, an eccentricrotating mass motor (“ERM”), a linear resonant actuator (“LRA”), a highbandwidth actuator, an electroactive polymer (“EAP”) actuator, anelectrostatic friction display, an ultrasonic vibration generator, anelectrostatic friction (“ESF”) device, and ultrasonic surface friction(“USF”) or a combination thereof.
 6. The computer implemented method ofclaim 5, wherein the specific haptic feedback device has a form factorto be worn on a wrist of the user.
 7. The computer implemented method ofclaim 2, wherein the analyzing of the content of the communicationssession from the content received by the conference system through theuser specific communications device for the at least one of the userscomprises: capturing a phone conversation for all participants from saiduser specific communication device; and measuring a volume for saidparticipants from said user specific communication device.
 8. Thecomputer implemented method of claim 7, wherein the measuring of thevolume comprises determining for said participants whether a said userspecific communication device is on mute.
 9. The computer implementedmethod of claim 7, wherein capturing status for said at least two usersfrom data measured by the user specific conduct measuring device for theat least two users comprises determining with the camera of the userspecific conduct measuring device a direction of users while talkingwith respect to the user specific communication device, determining adistance of users while talking with respect to the user specificcommunication device, determining whether user is speaking with anotherparty directly while the multi-party phone conversation is pending or acombination thereof.
 10. The computer implemented method of claim 2,wherein said determining with the conference system if the content ofthe communications session and the status of said at least two userscalls for input by the user through said user specific communicationsdevice comprises determining that the user is attempting to contributeto the multi-party phone conversation by speaking into the user specificcommunication device while it is set on said mute.
 11. The computerimplemented method of claim 1, wherein said determining with theconference system if the content of the communications session and thestatus of said at least two users calls for input by the user throughsaid user specific communications device comprises determining that thecommunications session includes the name of an individual user,determining the individual user is not facing said user specificcommunications device with the user specific conduct measuring device,and issuing said calls for input by the user.
 12. The computerimplemented method of claim 2, wherein said determining with theconference system if the content of the communications session and thestatus of said at least two users calls for input by the user throughsaid user specific communications device comprises determining that thecommunications session includes background noise not relevant to themulti-party conversation being measured from the user specificcommunications device, determining the specific communications device isnot muted; and issuing said calls for input to the user corresponding tothe specific communications device.
 13. The computer implemented methodof claim 12, wherein the back ground noise can be conversation with auser and an person that is not a party to the communications session.14. A system for providing feedback to participants of multi-party phoneconversations comprising: a registration database that includes aplurality of user accounts including addresses for user specificcommunications device registered with the conference system, a userspecific conduct measuring devices, and a user specific haptic feedbackdevices; a content input recorder for receiving communications contentfrom a communications session for a conversation between at least twousers from content received through the user specific communicationsdevice for at least two of the users in the registration database; aconduct input recorder for recording a status for said at least twousers from data measured by the user specific conduct measuring device,the status indicating the level of involvement of the at least two usersin the conversation; an individual status calculator including at leastone hardware device processor for performing a set of instruction storedon at least one memory device, the individual status calculatoranalyzing the content of the conversation of the communications sessionbetween the at least two users received from the user specificcommunications device from the at least two users, analyzing the statusof said at least two users measured by the user specific conductmeasuring device signals, and calculating whether an individual statusfor one of said at least two users calls for an input through said userspecific communications device by said user having the individual statuscalling for said input; and a feedback signal transmitter that sends asignal for input through said user specific communications device to auser having the individual status calling for said input, the userspecific haptic feedback device delivering the feedback signal to theuser having the individual status calling for said input.
 15. The systemof claim 14, wherein the specific communications device includes atelephone with a mute function, and wherein the specific conductmeasuring device is an internet of things (IOT) device including atleast one of a camera and a microphone.
 16. The system of claim 14,wherein the user specific haptic feedback device asserts a signal byphysical interaction with the user by an electric motor, anelectro-magnetic actuator, a linear resonant actuator, a piezoelectricactuator, a shape memory alloy, an electro-active polymer, a solenoid,an eccentric rotating mass motor (“ERM”), a linear resonant actuator(“LRA”), a high bandwidth actuator, an electroactive polymer (“EAP”)actuator, an electrostatic friction display, an ultrasonic vibrationgenerator, an electrostatic friction (“ESF”) device, and ultrasonicsurface friction (“USF”) or a combination thereof.
 17. A computerprogram product comprising a non-transitory computer readable storagemedium having computer readable program code embodied therein forproviding haptic feedback to participants of multi-party phoneconversations, the method comprising: opening a communications sessionwith a conference system for a conversation between at least two users,wherein each of the at least two users has a user specificcommunications device registered with the conference system over whichthe conversation is being transmitted, a user specific conduct measuringdevice registered with the conference system, and a user specific hapticfeedback device registered with the conference system; analyzing contentof the conversation of the communications session between the at leasttwo users from content received by the conference system through theuser specific communications device for at least one of the users;capturing status for said at least two users from data measured by theuser specific conduct measuring device for the at least two users, thestatus indicating the level of involvement of the at least two users inthe conversation; determining with the conference system if the contentof the communications session and the status of said at least two userscalls for input by the user to the conversation through said userspecific communications device; and sending a feedback signal from theconference system to the user specific haptic feedback device of theuser that the conference system has determined that the communicationssession has called for said input, the user specific haptic feedbackdevice delivering the feedback signal to the user.
 18. The computerprogram product of claim 17, wherein the specific communications deviceincludes a telephone with a mute function.
 19. The computer programproduct of claim 18, wherein the analyzing of the content of thecommunications session from the content received by the conferencesystem through the user specific communications device for the at leastone of the users comprises: capturing a phone conversation for allparticipants from said user specific communication device; and measuringa volume for said participants from said user specific communicationdevice.
 20. The computer program product of claim 18, wherein capturingstatus for said at least two users from data measured by the userspecific conduct measuring device for the at least two users comprisesdetermining with the camera of the user specific conduct measuringdevice a direction of users while talking with respect to the userspecific communication device, determining a distance of users whiletalking with respect to the user specific communication device,determining whether user is speaking with another party directly whilethe multi-party phone conversation is pending or a combination thereof.